Papers with large-scale pretrained language models
A Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations, but they are plagued by the Knowledge Hallucination problem. |
| Approach: | They propose a method that exploits the dialogue-knowledge interaction to reduce hallucination by using external knowledge resources to generate more informative responses. |
| Outcome: | The proposed method reduces hallucination without disrupting other dialogue performance while keeping adaptive to different generation models. |
Scientific and Creative Analogies in Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing analogy datasets focus on a limited set of analogical relations with a high similarity of the two domains between which the analogy holds. |
| Approach: | They propose a dataset that encodes analogy in pretrained language models . they use a system that maps attributes and relational structures across dissimilar domains . |
| Outcome: | The proposed dataset shows that state-of-the-art models achieve low performance on analogy tasks . |
Robust Navigation with Language Pretraining and Stochastic Sampling (D19-1)
Copied to clipboard
Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah A. Smith, Yejin Choi
| Challenge: | Existing methods to learn visual representations and action decoding schemes are limited to previously unseen instructions and environments. |
| Approach: | They propose a stochastic sampling scheme to reduce the gap between the expert actions in training and sampled actions in test to correct its own mistakes. |
| Outcome: | The proposed methods achieve 6% absolute gain over the previous best results on the Room-to-Room benchmark. |
LMSOC: An Approach for Socially Sensitive Pretraining (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretraining models have been shown to learn effective linguistic representations for many NLP tasks, but there are many real-world contextual aspects of language that current approaches do not capture. |
| Approach: | They propose to integrate speaker social context into the learned representations of large-scale language models by using graph representation learning algorithms and primed language model pretraining with these social context representations. |
| Outcome: | The proposed approach improves on geographically sensitive language modeling tasks by more than 100% relative lift on MRR compared to baselines. |
IGA: An Intent-Guided Authoring Assistant (2021.emnlp-main)
Copied to clipboard
Simeng Sun, Wenlong Zhao, Varun Manjunatha, Rajiv Jain, Vlad Morariu, Franck Dernoncourt, Balaji Vasan Srinivasan, Mohit Iyyer
| Challenge: | Pretrained language models have improved writing assistance functions such as autocomplete, but more complex and controllable writing assistants have yet to be explored. |
| Approach: | They build an intent-guided authoring assistant that follows fine-grained author directives by specifying different writing intents. |
| Outcome: | The proposed system generates output satisfying the author's intent and can be rephrased to their liking. |
Selective Token Generation for Few-shot Natural Language Generation (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that the proposed selective token generation algorithm outperforms the previous additive learning algorithms based on the PLMs. |
| Approach: | They propose an additive learning algorithm that selectively outputs language tokens between a task-general PLM and a specific adapter during training and inference. |
| Outcome: | The proposed algorithm outperforms existing methods on few-shot natural language generation tasks. |
Entity-based Neural Local Coherence Modeling (2022.acl-long)
Copied to clipboard
| Challenge: | Recent neural coherence models encode the input document using large-scale pretrained language models. |
| Approach: | They propose an entity-based neural local coherence model which is linguistically more sound than previous models. |
| Outcome: | The proposed model outperforms existing models on three downstream tasks. |